For gathering dimensional information about an object, particularly of topographical information, the surface of such object typically may be scanned using a laser beam which is moved over the object in a predefined manner. Precise scanning of the object for instance is provided by a geodetic measuring device like a terrestrial laser scanner or a total station, e.g. Leica P20, Leica Multi Station 50. By scanning the object a so called (3D-) point cloud is created representing the object by an amount of points with defined positions in a common coordinate system.
The point cloud is derived by determining a distance for each measuring point and a correlated direction of the laser beam when determining the distance. The point-to-point resolution of such measuring points and the resulting point cloud, respectively, is defined by the speed of moving the laser beam on the surface and a triggering-interval for triggering single measurements (e.g. one for each measuring point).
In addition to generating the point cloud there often is captured an image of the object by a capturing unit of the geodetic measuring device. The image provides further information concerning the object, e.g. concerning colours or textures of the object.
As from one station point usually only a part of the object is measurable while other surface points are hidden, it becomes necessary to set up the measuring devices at least at two different positions with respect to the object such that in combination the whole surface of the object is measurable.
The surveying instrument needs direct line-of-sight to the object points to measure. In case of an obstruction, e.g. a tree in front of a building which occludes a part of the façade leads to a so called “scanning shadow” (see FIG. 1a). In practice, in such a case the surveying instrument also is set up at a different position where direct line-of-sight to the missing parts is given. Therefore, more than one setup of the surveying instruments is needed and each additional setup takes time and reduces the productivity of the user.
Moreover, a full-dome-scan, i.e. a scanning area from 0° to 360° in horizontal and −45° to 90° in vertical direction, with a terrestrial laser scanner in highest resolution may take up to several hours. In this resolution the distance between the points in 100 meters is 1.0 mm. For every new setup of the instrument a full 360° panorama image is usually obtained which also takes several minutes. Thus, relocating a laser scanner or a similar surveying instrument (e.g. total station) and recording a second set of measuring data (second point cloud) is very time consuming and needs an expert at least for referencing the first point cloud relative to the second point cloud.
EP 1 903 303 B1 discloses a method of combining point cloud data with image data in order to fill up missing parts of the point cloud. The camera unit is used for recording a set of images which are split into a set of stereoscopic image pairs. Every image pair is processed independently. Moreover, the panorama image obtained by a laser scanner (the so-called “main image”) is used for pair wise matching with one stereoscopic image pair and thus providing adding dimensional information of the respective stereoscopic image pair to the point cloud. The whole process is performed in a post-processing step having all data of the set of images and the laser scanner ready for processing.
Several below mentioned methods for registration of point clouds are known, e.g. marker-based or image-based registration first for coarse registration and afterwards geometry-based registration (iterative closest point) for fine adjustment:
Marker-Based Registration:
A user places three or more markers around an object. The positions of the markers are measured with the surveying instrument—either by a reflector-less measurement onto the marker or with a surveying pole or by scanning.
These markers are scanned using a laser scanner or total station and also each marker should be visible at least on one image or video frame from the mobile camera unit. Such coded markers are detected automatically (preferred), alternatively manually, in laser-based point cloud or panorama image and on images from the mobile camera unit.
Correspondences between marker positions in image-based and laser-based coordinate frames are used for registration.
In case of non-coded markers the identification of corresponding markers positions can be carried out automatically by analyzing the distances between marker pairs.
Image-Based Registration:
The registration task might be solved by finding a set of common points in images, using image processing. The fact corresponding points on images are related with 3D points is used to compute Helmert transformation. At least three of the measured image points should have corresponding 3D points in the associated point clouds.
Geometry-Based Registration:
The registration task might be solved by finding a set of common points in point clouds e.g. using 3D feature descriptors. Alternatively, a geometry based method might be used for registration. Two registered point clouds should cover overlapping object surfaces. Such methods are comparing geometric similarity between point clouds to find an optimal transformation.
Manual Registration:
Here, three or more common points in image-based and laser-based point cloud are selected manually, e.g. by displaying the point clouds and enabling the user to pick the corresponding points. The registration task is solved by computing of rigid motion (or Helmert) transformation using common point correspondences.
A main disadvantage of above method is that due to the post-processing and due to the independent processing of the stereoscopic images an error concerning the accuracy of point positions increases with the number of images not being directly related to the scanning point cloud. Moreover, a compensation of position errors emanating from the stereoscopic image processing subsequent to the capturing process is not possible or possible only to a minor. Additionally, the processing according to the state of the art needs to be performed in a post-processing manner, i.e. not in the field directly in the course of the measurement, and thus, the user cannot be provided with actual information regarding e.g. a quality of the performed measurements.